12 Data Collection Techniques

In this column we provide a context for many of the types of data collection techniques used with human participants. We will not discuss methods for assessing diagnostic status, but we will provide some information about developing or evaluating a questionnaire, test, or other data collection technique. Research approaches or designs are approximately orthogonal to the techniques of data collection, and thus, in theory, any type of data collection technique could be used with any approach to research. However, some types of data collection are more commonly used with the experimental approaches. Others are more common with comparative or associational (survey) approaches, and still others are more common in qualitative research. Table 1 gives an approximation of how common each of several data collection techniques are within each of these three major groupings of research approaches. Note that we have ordered the data collection techniques along a dimension from observer/researcher report to self-report measures. The observer report end includes observations and physiological recordings that are probably less influenced by the participants’ desire to look good, but they are affected by any biases the observer may have. Of course, if the participants realize that they are being observed, they may not behave naturally. At the other end of this dimension are measures based on self-reports of the participants, such as interviews and questionnaires. In these cases, responses are certainly filtered through the participants’ eyes and are probably heavily influenced by factors such as social desirability. Concern about faulty memories or socially desirable responses lead researchers, especially those who use experiments, to be suspicious about the validity of the self-reports. On the other hand, observer reports are not necessarily valid measures. For example, qualitative researchers point out that cultural biases may lead observers to misinterpret their observations. In general, it is advisable to select instruments that have been used in other studies if they have been shown to be reliable and valid with the planned types of participants and for purposes similar to that for the planned study.